Control device, control method, projection system, and control program

ABSTRACT

A control device of a projection system includes one or more projection apparatuses that project a first image including a plurality of marker images, and an imaging apparatus that captures at least a part of the first image, the control device includes a processor, and the processor is configured to perform a control of projecting a second image including a plurality of marker images from the projection apparatus based on a capturing result of at least the part of the first image by the imaging apparatus.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority under 35 U.S.C. § 119 toJapanese Patent Application No. 2021-126129, filed on Jul. 30, 2021.This Japanese Patent Application is hereby expressly incorporated byreference, in its entirety, into the present application.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to a control device, a control method, aprojection system, and a computer readable medium storing a controlprogram.

2. Description of the Related Art

JP2015-026992A discloses capturing calibration projection imagesprojected from a plurality of projectors separately in a plurality ofregions by a camera and calculating various correction coefficients forperforming registration, scale matching, distortion correction,brightness correction of an overlapping region, and the like for eachprojector.

JP2016-204068A1 discloses capturing a projection region of a projectorseparately in a plurality of regions in a partially overlapping mannerand estimating a projective transformation matrix for connectingadjacent partial images to each other based on a captured image.

JP2012-047849A discloses, in stack projection, projecting overlappingtest patterns at the same time and capturing the test patterns,projecting patterns obtained by changing a wavelength region such as R,and B or patterns obtained by changing polarization characteristics foreach projector, and retrospectively separating the overlapping patterns.

SUMMARY OF THE INVENTION

One embodiment according to the disclosed technology provides a controldevice, a control method, a projection system, and a computer readablemedium storing a control program that can easily adjust projection of aprojection apparatus with respect to a wide projection range.

A control device according to an aspect of the present invention is acontrol device of a projection system including one or more projectionapparatuses that project a first image including a plurality of markerimages, and an imaging apparatus that captures at least a part of thefirst image, the control device comprising a processor, in which theprocessor is configured to perform a control of projecting a secondimage including a plurality of marker images from the projectionapparatus based on a capturing result of at least the part of the firstimage by the imaging apparatus.

A control method according to another aspect of the present invention isa control method by a control device of a projection system includingone or more projection apparatuses that project a first image includinga plurality of marker images, and an imaging apparatus that captures atleast a part of the first image, the control device including aprocessor, the control method comprising performing, by the processor, acontrol of projecting a second image including a plurality of markerimages from the projection apparatus based on a capturing result of atleast the part of the first image by the imaging apparatus.

A projection system according to still another aspect of the presentinvention is a projection system comprising one or more projectionapparatuses that project a first image including a plurality of markerimages, an imaging apparatus that captures at least a part of the firstimage, and a control device, in which the control device includes aprocessor, and the processor is configured to perform a control ofprojecting a second image including a plurality of marker images fromthe projection apparatus based on a capturing result of at least thepart of the first image by the imaging apparatus.

A control program according to still another aspect of the presentinvention causes a processor of a control device of a projection systemto execute a process, the projection system including one or moreprojection apparatuses that project a first image including a pluralityof marker images, and an imaging apparatus that captures at least a partof the first image, the process comprising performing a control ofprojecting a second image including a plurality of marker images fromthe projection apparatus based on a capturing result of at least thepart of the first image by the imaging apparatus.

According to the present invention, a control device, a control method,a projection system, and a control program that can easily adjustprojection of a projection apparatus with respect to a wide projectionrange can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of a projection system 100of an embodiment.

FIG. 2 is a diagram illustrating an example of a projection apparatus10.

FIG. 3 is a schematic diagram illustrating an example of an internalconfiguration of a projection portion 1.

FIG. 4 is a schematic diagram illustrating an exterior configuration ofthe projection apparatus 10.

FIG. 5 is a schematic cross-sectional view of an optical unit 106 of theprojection apparatus 10 illustrated in FIG. 4 .

FIG. 6 is a diagram illustrating an example of a hardware configurationof a computer 50.

FIG. 7 is a diagram illustrating an example of projection of a firstimage by the projection apparatus 10 and an imageable range of animaging apparatus 90.

FIG. 8 is a diagram (Part 1) illustrating an example of projection of asecond image by the projection apparatus 10.

FIG. 9 is a diagram (Part 2) illustrating the example of projection ofthe second image by the projection apparatus 10.

FIG. 10 is a diagram (Part 3) illustrating the example of projection ofthe second image by the projection apparatus 10.

FIG. 11 is a diagram (Part 4) illustrating the example of projection ofthe second image by the projection apparatus 10.

FIG. 12 is a flowchart illustrating an example of processing by thecomputer 50.

FIG. 13 is a diagram illustrating another example of a first image 71projected by the projection apparatus 10.

FIG. 14 is a diagram illustrating another example of a second image 81projected by the projection apparatus 10.

FIG. 15 is a diagram illustrating an example of a state beforeadjustment of stack projection by a plurality of projection apparatuses.

FIG. 16 is a flowchart illustrating an example of processing by thecomputer 50 based on an imaging condition.

FIG. 17 is a diagram (Part 1) illustrating an example of individualprojection of projection apparatuses 10 and 10A.

FIG. 18 is a diagram (Part 2) illustrating the example of the individualprojection of the projection apparatuses 10 and 10A.

FIG. 19 is a diagram illustrating an example of projection of theprojection apparatuses 10 and 10A at the same time.

FIG. 20 is a diagram illustrating an example of the stack projection bymaking projection ranges 11 and 11A overlap.

FIG. 21 is a diagram illustrating an example of a state beforeadjustment of blending projection by the plurality of projectionapparatuses.

FIG. 22 is a diagram illustrating an example of the blending projectionby making the projection ranges 11 and 11A overlap.

FIG. 23 is a flowchart illustrating another example of the processing bythe computer 50 based on the imaging condition.

FIG. 24 is a diagram illustrating an example of the second image in acase where a resolution of imaging of the imaging apparatus 90 is low.

FIG. 25 is a flowchart illustrating still another example of theprocessing by the computer 50 based on the imaging condition.

FIG. 26 is a diagram illustrating an example of marker images of thesecond image in a case where the resolution of the imaging of theimaging apparatus 90 is low.

FIG. 27 is a schematic diagram illustrating another exteriorconfiguration of the projection apparatus 10.

FIG. 28 is a schematic cross-sectional view of the optical unit 106 ofthe projection apparatus 10 illustrated in FIG. 27 .

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an example of an embodiment of the present invention willbe described with reference to the drawings.

EMBODIMENT

Projection System 100 of Embodiment

FIG. 1 is a diagram illustrating an example of a projection system 100of the embodiment. As illustrated in FIG. 1 , the projection system 100comprises a projection apparatus 10, a computer 50, and an imagingapparatus 90. The computer 50 is an example of a control deviceaccording to the embodiment of the present invention.

The computer 50 can communicate with the projection apparatus 10 and theimaging apparatus 90. In the example illustrated in FIG. 1 , thecomputer 50 is connected to the projection apparatus 10 through acommunication cable 8 and can communicate with the projection apparatus10. The computer 50 is connected to the imaging apparatus 90 through acommunication cable 9 and can communicate with the imaging apparatus 90.

The projection apparatus 10 is a projection apparatus that can performprojection to a projection target object 6. The imaging apparatus 90 isan imaging apparatus that can capture an image projected to theprojection target object 6 by the projection apparatus 10.

The projection target object 6 is an object such as a screen having aprojection surface on which a projection image is displayed by theprojection apparatus 10. In the example illustrated in FIG. 1 , theprojection surface of the projection target object 6 is a rectangularplane. It is assumed that upper, lower, left, and right sides of theprojection target object 6 in FIG. 1 are upper, lower, left, and rightsides of the actual projection target object 6.

A projection range 11 illustrated by a dot dashed line is a region thatis irradiated with projection light by the projection apparatus 10 inthe projection target object 6. In the example illustrated in FIG. 1 ,the projection range 11 is rectangular. The projection range 11 is apart or the entirety of a projectable range within which the projectioncan be performed by the projection apparatus 10.

Projection Apparatus 10

FIG. 2 is a diagram illustrating an example of the projection apparatus10. As illustrated in FIG. 2 , the projection apparatus 10 comprises aprojection portion 1, a control portion 4, an operation receptionportion 2, and a communication portion 5. The projection portion 1 isconfigured with, for example, a liquid crystal projector or a projectorusing liquid crystal on silicon (LCOS). Hereinafter, the projectionportion 1 will be described as a liquid crystal projector.

The control portion 4 controls the projection performed by theprojection apparatus 10. The control portion 4 is a device including acontrol portion configured with various processors, a communicationinterface (not illustrated) for communicating with each portion, and astorage medium 4 a such as a hard disk, a solid state drive (SSD), or aread only memory (ROM) and generally controls the projection portion 1.Examples of the various processors of the control portion of the controlportion 4 include a central processing unit (CPU) that is ageneral-purpose processor performing various processing by executing aprogram, a programmable logic device (PLD) such as a field programmablegate array (FPGA) that is a processor having a circuit configurationchangeable after manufacturing, or a dedicated electric circuit such asan application specific integrated circuit (ASIC) that is a processorhaving a circuit configuration dedicatedly designed to execute specificprocessing.

More specifically, a structure of these various processors is anelectric circuit in which circuit elements such as semiconductorelements are combined. The control portion of the control portion 4 maybe configured with one of the various processors or may be configuredwith a combination of two or more processors of the same type ordifferent types (for example, a combination of a plurality of FPGAs or acombination of a CPU and an FPGA).

The operation reception portion 2 detects an instruction (userinstruction) from a user by receiving various operations from the user.The operation reception portion 2 may be a button, a key, a joystick, orthe like provided in the control portion 4 or a reception portion or thelike that receives a signal from a remote controller for remotelyoperating the control portion 4.

The communication portion 5 is a communication interface that cancommunicate with the computer 50. The communication portion 5 may be awired communication interface that performs wired communication asillustrated in FIG. 1 , or a wireless communication interface thatperforms wireless communication.

The projection portion 1, the control portion 4, and the operationreception portion 2 are implemented by, for example, one device (forexample, refer to FIG. 4 and FIG. 5 ). Alternatively, the projectionportion 1, the control portion 4, and the operation reception portion 2may be separate devices that cooperate by communicating with each other.

Internal Configuration of Projection Portion 1

FIG. 3 is a schematic diagram illustrating an example of an internalconfiguration of the projection portion 1. As illustrated in FIG. 3 ,the projection portion 1 comprises a light source 21, a light modulationportion 22, a projection optical system 23, and a control circuit 24.The light source 21 includes a light emitting element such as a laser ora light emitting diode (LED) and emits, for example, white light.

The light modulation portion 22 is configured with three liquid crystalpanels (light modulation elements) that emit each color image bymodulating, based on image information, each color light which isemitted from the light source 21 and is separated into three colors ofred, blue, and green by a color separation mechanism, not illustrated,and a dichroic prism that mixes each color image emitted from the threeliquid crystal panels and emits the mixed color image in the samedirection. Filters of red, blue, and green may be mounted in each of thethree liquid crystal panels, and each color image may be emitted bymodulating the white light emitted from the light source 21 in eachliquid crystal panel.

The light from the light source 21 and the light modulation portion 22is incident on the projection optical system 23. The projection opticalsystem 23 includes at least one lens and is composed of, for example, arelay optical system. The light that has passed through the projectionoptical system 23 is projected to the projection target object 6.

In the projection target object 6, a region irradiated with the lighttransmitted through the entire range of the light modulation portion 22is the projectable range within which the projection can be performed bythe projection portion 1. In the projectable range, a region that isactually irradiated with the light transmitted through the lightmodulation portion 22 is the projection range 11. For example, in theprojectable range, a size, a position, and a shape of the projectionrange 11 are changed by controlling a size, a position, and a shape of aregion through which the light is transmitted in the light modulationportion 22.

The control circuit 24 projects an image based on display data to theprojection target object 6 by controlling the light source 21, the lightmodulation portion 22, and the projection optical system 23 based on thedisplay data input from the control portion 4. The display data inputinto the control circuit 24 is configured with three constituents of reddisplay data, blue display data, and green display data.

In addition, the control circuit 24 enlarges or reduces the projectionrange 11 (refer to FIG. 1 ) of the projection portion 1 by changing theprojection optical system 23 based on an instruction input from thecontrol portion 4. In addition, the control portion 4 may move theprojection range 11 of the projection portion 1 by changing theprojection optical system 23 based on an operation received by theoperation reception portion 2 from the user.

In addition, the projection apparatus 10 comprises a shift mechanismthat mechanically or optically moves the projection range 11 whilemaintaining an image circle of the projection optical system 23. Theimage circle of the projection optical system 23 is a region in whichthe projection light incident on the projection optical system 23appropriately passes through the projection optical system 23 in termsof a light fall-off, color separation, edge part curvature, or the like.

The shift mechanism is implemented by at least one of an optical systemshift mechanism that performs optical system shifting, or an electronicshift mechanism that performs electronic shifting.

The optical system shift mechanism is, for example, a mechanism (forexample, refer to FIG. 5 and FIG. 28 ) that moves the projection opticalsystem 23 in a direction perpendicular to an optical axis, or amechanism that moves the light modulation portion 22 in the directionperpendicular to the optical axis instead of moving the projectionoptical system 23. In addition, the optical system shift mechanism mayperform the movement of the projection optical system 23 and themovement of the light modulation portion 22 in combination.

The electronic shift mechanism is a mechanism that performs pseudoshifting of the projection range 11 by changing a range through whichthe light is transmitted in the light modulation portion 22.

In addition, the projection apparatus 10 may comprise a projectiondirection changing mechanism that moves the image circle of theprojection optical system 23 and the projection range 11. The projectiondirection changing mechanism is a mechanism that changes a projectiondirection of the projection portion 1 by changing a direction of theprojection portion 1 by mechanical rotation (for example, refer to FIG.28 ).

Mechanical Configuration of Projection Apparatus 10

FIG. 4 is a schematic diagram illustrating an exterior configuration ofthe projection apparatus 10. FIG. 5 is a schematic cross-sectional viewof an optical unit 106 of the projection apparatus 10 illustrated inFIG. 4 . FIG. 5 illustrates a cross section in a plane along an opticalpath of light emitted from a body part 101 illustrated in FIG. 4 .

As illustrated in FIG. 4 , the projection apparatus 10 comprises thebody part 101 and the optical unit 106 that is provided to protrude fromthe body part 101. In the configuration illustrated in FIG. 4 , theoperation reception portion 2; the control portion 4; the light source21, the light modulation portion 22, and the control circuit 24 in theprojection portion 1; and the communication portion 5 are provided inthe body part 101. The projection optical system 23 in the projectionportion 1 is provided in the optical unit 106.

The optical unit 106 comprises a first member 102 supported by the bodypart 101. The optical unit 106 may be configured to be attachable to anddetachable from the body part 101 (in other words, interchangeablyconfigured).

As illustrated in FIG. 5 , the body part 101 includes a housing 15 inwhich an opening 15 a for passing light is formed in a part connected tothe optical unit 106.

As illustrated in FIG. 4 , the light source 21 and a light modulationunit 12 including the light modulation portion 22 (refer to FIG. 3 )that generates an image by spatially modulating the light emitted fromthe light source 21 based on input image data are provided inside thehousing 15 of the body part 101. The light emitted from the light source21 is incident on the light modulation portion 22 of the lightmodulation unit 12 and is spatially modulated and emitted by the lightmodulation portion 22.

As illustrated in FIG. 5 , the image formed by the light spatiallymodulated by the light modulation unit 12 is incident on the opticalunit 106 through the opening 15 a of the housing 15 and is projected tothe projection target object 6. Accordingly, an image G1 is visible froman observer.

As illustrated in FIG. 5 , the optical unit 106 comprises the firstmember 102 having a hollow portion 2A connected to an inside of the bodypart 101, a first optical system 121 arranged in the hollow portion 2A,a lens 34, and a first shift mechanism 105.

The first member 102 is a member having, for example, a rectangularcross-sectional exterior, in which an opening 2 a and an opening 2 b areformed in surfaces parallel to each other. The first member 102 issupported by the body part 101 in a state where the opening 2 a isarranged at a position facing the opening 15 a of the body part 101. Thelight emitted from the light modulation portion 22 of the lightmodulation unit 12 of the body part 101 is incident into the hollowportion 2A of the first member 102 through the opening 15 a and theopening 2 a.

An incidence direction of the light incident into the hollow portion 2Afrom the body part 101 will be referred to as a direction X1. Adirection opposite to the direction X1 will be referred to as adirection X2. The direction X1 and the direction X2 will be collectivelyreferred to as a direction X. In addition, in FIG. 5 , a direction fromthe front to the back of the page and an opposite direction will bereferred to as a direction Z. In the direction Z, the direction from thefront to the back of the page will be referred to as a direction Z1, andthe direction from the back to the front of the page will be referred toas a direction Z2.

In addition, a direction perpendicular to the direction X and thedirection Z will be referred to as a direction Y. In the direction Y, anupward direction in FIG. 5 will be referred to as a direction Y1, and adownward direction in FIG. 5 will be referred to as a direction Y2. Inthe example in FIG. 5 , the projection apparatus 10 is arranged suchthat the direction Y2 is a vertical direction.

The projection optical system 23 illustrated in FIG. 3 is composed ofthe first optical system 121 and the lens 34 in the example in FIG. 5 .An optical axis K of this projection optical system 23 is illustrated inFIG. 5 . The first optical system 121 and the lens 34 are arranged inthis order from the light modulation portion 22 side along the opticalaxis K.

The first optical system 121 includes at least one lens and guides thelight that is incident on the first member 102 from the body part 101and travels in the direction X1, to the lens 34.

The lens 34 is arranged in an end part of the first member 102 on thedirection X1 side in the form of closing the opening 2 b formed in thisend part. The lens 34 projects the light incident from the first opticalsystem 121 to the projection target object 6.

The first shift mechanism 105 is a mechanism for moving the optical axisK of the projection optical system (in other words, the optical unit106) in a direction (direction Y in FIG. 5 ) perpendicular to theoptical axis K. Specifically, the first shift mechanism 105 isconfigured to be capable of changing a position of the first member 102in the direction Y with respect to the body part 101. The first shiftmechanism 105 may manually move the first member 102 or electricallymove the first member 102.

FIG. 5 illustrates a state where the first member 102 is moved as far aspossible to the direction Y1 side by the first shift mechanism 105. Bymoving the first member 102 in the direction Y2 by the first shiftmechanism 105 from the state illustrated in FIG. 5 , a relative positionbetween a center of the image (in other words, a center of a displaysurface) formed by the light modulation portion 22 and the optical axisK changes, and the image G1 projected to the projection target object 6can be shifted (translated) in the direction Y2.

The first shift mechanism 105 may be a mechanism that moves the lightmodulation portion 22 in the direction Y instead of moving the opticalunit 106 in the direction Y. Even in this case, the image G1 projectedto the projection target object 6 can be moved in the direction Y.

Hardware Configuration of Computer 50

FIG. 6 is a diagram illustrating an example of a hardware configurationof the computer 50. As illustrated in FIG. 6 , the computer 50illustrated in FIG. 1 comprises a processor 51, a memory 52, acommunication interface 53, and a user interface 54. For example, theprocessor 51, the memory 52, the communication interface 53, and theuser interface 54 are connected by a bus 59.

For example, the processor 51 is a circuit performing signal processingand is a CPU that controls the entire computer 50. The processor 51 maybe implemented by other digital circuits such as an FPGA and a digitalsignal processor (DSP). In addition, the processor 51 may be implementedby combining a plurality of digital circuits.

For example, the memory 52 includes a main memory and an auxiliarymemory. For example, the main memory is a random access memory (RAM).The main memory is used as a work area of the processor 51.

For example, the auxiliary memory is a non-volatile memory such as amagnetic disk, an optical disc, or a flash memory. The auxiliary memorystores various programs for operating the computer 50. The programsstored in the auxiliary memory are loaded into the main memory andexecuted by the processor 51.

In addition, the auxiliary memory may include a portable memory that canbe detached from the computer 50. Examples of the portable memoryinclude a memory card such as a universal serial bus (USB) flash driveor a secure digital (SD) memory card and an external hard disk drive.

The communication interface 53 is a communication interface thatcommunicates with an outside (for example, the projection apparatus 10and the imaging apparatus 90) of the computer 50. The communicationinterface 53 is controlled by the processor 51. The communicationinterface 53 may be a wired communication interface that performs wiredcommunication or a wireless communication interface that performswireless communication or may include both of the wired communicationinterface and the wireless communication interface.

For example, the user interface 54 includes an input device thatreceives an operation input from a user, and an output device thatoutputs information to the user. For example, the input device can beimplemented by a pointing device (for example, a mouse), a key (forexample, a keyboard), or a remote controller. For example, the outputdevice can be implemented by a display or a speaker. In addition, theinput device and the output device may be implemented by a touch panelor the like. The user interface 54 is controlled by the processor 51.

Projection of First Image by Projection Apparatus 10 and Imageable Rangeof Imaging Apparatus 90

FIG. 7 is a diagram illustrating an example of projection of a firstimage by the projection apparatus 10 and an imageable range of theimaging apparatus 90. As illustrated in FIG. 7 , the computer 50performs a control of projecting a first image 71 to the projectionrange 11 from the projection apparatus 10. The first image 71 is animage in which 91 rectangular marker images are arranged in a 7×13matrix.

An imageable range 72 is a range that can be imaged by the imagingapparatus 90 in the projection target object 6. As illustrated in FIG. 7, the imageable range 72 may be narrower than the projection range 11,and the entire first image 71 may not be imageable by the imagingapparatus 90. For example, this is because of restrictions on an angleof view of the imaging apparatus 90 or restrictions on a distance inwhich the imaging apparatus 90 can be separated from the projectiontarget object 6.

For example, the computer 50 performs a control of outputting a messagethat prompts a user of the imaging apparatus 90 to perform imaging byincluding as many marker images as possible among the marker images ofthe first image 71. The user of the imaging apparatus 90 may be the sameas or different from the user of the computer 50.

The user of the imaging apparatus 90 captures at least a part of thefirst image 71 by the imaging apparatus 90. In the example in FIG. 7 ,it is assumed that the entire first image 71 cannot be captured by theimaging apparatus 90, and imaging is performed by causing a region thatis a part near a center of the first image 71 and includes 4×7=28 markerimages to fall within the imageable range 72. The imaging apparatus 90transmits a captured image obtained by the imaging to the computer 50.

The computer 50 calculates the imageable range 72 based on the capturedimage transmitted from the imaging apparatus 90. Specifically, thecalculation of the imageable range 72 is calculation of a relative sizeof the imageable range 72 with respect to the first image 71 projectedto the projection target object 6. For example, the computer 50calculates the number of marker images (in the example in FIG. 7, 28 )included in the captured image from the imaging apparatus 90 among themarker images included in the first image 71 as the size of theimageable range 72. The computer 50 generates a second image including aplurality of marker images included in the imageable range 72 based on acalculation result of the imageable range 72.

For example, the computer 50 determines the number of marker imagesincluded in the captured image from the imaging apparatus 90 by imagerecognition. In the example in FIG. 7 , 4×7=28 marker images areincluded in the captured image from the imaging apparatus 90.Accordingly, the computer 50 generates the second image including 4×7=28marker images.

Projection of Second Images by Projection Apparatus 10

FIG. 8 to FIG. 11 are diagrams illustrating projection of the secondimage by the projection apparatus 10. After the imageable range 72 iscalculated as described using FIG. 7 , for example, the computer 50performs a control of projecting a second image 81 to an upper left partof the projection range 11 from the projection apparatus 10 asillustrated in FIG. 8 .

The second image 81 is obtained by extracting an upper left part of4×7=28 marker images from the first image 71. That is, the second image81 is an image in which 28 rectangular marker images are arranged in a4×7 matrix. In addition, sizes and intervals of the marker imagesincluded in the second image 81 are the same as sizes and intervals ofthe marker images included in the first image 71. Accordingly, the userof the imaging apparatus 90 can perform the imaging by the imagingapparatus 90 by causing the 28 marker images included in the secondimage 81 to fall within the imageable range 72. The imaging apparatus 90transmits a captured image of the second image 81 obtained by theimaging to the computer 50.

In the state illustrated in FIG. 8 , the computer 50 may perform acontrol of prompting the user of the imaging apparatus 90 to perform theimaging by including all marker images of the second image 81. In theexample in FIG. 8 , the computer 50 projects a message “Please performimaging by including upper left marker group” to a different positionfrom the second image 81 in the projection range 11 by controlling theprojection apparatus 10.

Next, for example, as illustrated in FIG. 9 , the computer 50 performs acontrol of projecting the second image 82 to an upper right part of theprojection range 11 from the projection apparatus 10. A second image 82is obtained by extracting an upper right part of 4×7=28 marker imagesfrom the first image 71. That is, the second image 82 is an image inwhich 28 rectangular marker images are arranged in a 4×7 matrix in thesame manner as the second image 81. Accordingly, the user of the imagingapparatus 90 can perform the imaging by the imaging apparatus 90 bycausing the 28 marker images included in the second image 82 to fallwithin the imageable range 72. The imaging apparatus 90 transmits acaptured image of the second image 82 obtained by the imaging to thecomputer 50.

In the state illustrated in FIG. 9 , the computer 50 may perform acontrol of prompting the user of the imaging apparatus 90 to perform theimaging by including all marker images of the second image 82. In theexample in FIG. 9 , the computer 50 projects a message “Please performimaging by including upper right marker group” to a different positionfrom the second image 82 in the projection range 11 by controlling theprojection apparatus 10.

Next, for example, as illustrated in FIG. 10 , the computer 50 performsa control of projecting a second image 83 to a lower left part of theprojection range 11 from the projection apparatus 10. The second image83 is obtained by extracting a lower left part of 4×7=28 marker imagesfrom the first image 71. That is, the second image 83 is an image inwhich 28 rectangular marker images are arranged in a 4×7 matrix in thesame manner as the second images 81 and 82. Accordingly, the user of theimaging apparatus 90 can perform the imaging by the imaging apparatus 90by causing the 28 marker images included in the second image 83 to fallwithin the imageable range 72. The imaging apparatus 90 transmits acaptured image of the second image 83 obtained by the imaging to thecomputer 50.

In the state illustrated in FIG. 10 , the computer 50 may perform acontrol of prompting the user of the imaging apparatus 90 to perform theimaging by including all marker images of the second image 83. In theexample in FIG. 10 , the computer 50 projects a message “Please performimaging by including lower left marker group” to a different positionfrom the second image 83 in the projection range 11 by controlling theprojection apparatus 10.

Next, for example, as illustrated in FIG. 11 , the computer 50 performsa control of projecting a second image 84 to a lower right part of theprojection range 11 from the projection apparatus 10. The second image84 is obtained by extracting a lower right part of 4×7=28 marker imagesfrom the first image 71. That is, the second image 84 is an image inwhich 28 rectangular marker images are arranged in a 4×7 matrix in thesame manner as the second images 81 to 83. Accordingly, the user of theimaging apparatus 90 can perform the imaging by the imaging apparatus 90by causing the 28 marker images included in the second image 84 to fallwithin the imageable range 72. The imaging apparatus 90 transmits acaptured image of the second image 84 obtained by the imaging to thecomputer 50.

In the state illustrated in FIG. 11 , the computer 50 may perform acontrol of prompting the user of the imaging apparatus 90 to perform theimaging by including all marker images of the second image 84. In theexample in FIG. 11 , the computer 50 projects a message “Please performimaging by including lower right marker group” to a different positionfrom the second image 84 in the projection range 11 by controlling theprojection apparatus 10.

As described using FIG. 8 to FIG. 11 , the computer 50 can obtaininformation equivalent to the captured image of the entire first image71 by receiving the captured images of the second images 81 to 84 fromthe imaging apparatus 90. The computer 50 performs a control ofadjusting the projection of the projection apparatus 10 based on thereceived captured images of the second images 81 to 84.

For example, the computer 50 generates an image equivalent to thecaptured image of the entire first image 71 by combining the receivedcaptured images of the second images 81 to 84. The computer 50 detectsdistortion of a projection image within the projection range 11 based ondistortion in shape or arrangement of marker images included in thegenerated image and performs distortion correction of the projectionimage based on a detection result.

The computer 50 causes the imaging apparatus 90 to capture at least apart of the first image 71 by projecting the first image 71 including aplurality of marker images from the projection apparatus 10 and repeatsthe control of projecting the second image (for example, the secondimages 81 to 84) including a plurality of marker images from theprojection apparatus 10 based on a capturing result.

Specifically, the computer 50 calculates the imageable range 72 of theimaging apparatus 90 based on a captured image of at least a part of thefirst image 71 and performs the control of projecting the second images81 to 84 from the projection apparatus 10 based on the calculatedimageable range 72. At this point, the computer 50 projects the secondimages 81 to 84 from the projection apparatus 10 while changing aprojection position of each of the second images 81 to 84. The computer50 performs the control of adjusting the projection of the projectionapparatus 10 based on capturing results of the second images 81 to 84 bythe imaging apparatus 90.

Accordingly, even in a case where the entire first image 71 projected tothe projection range 11 cannot be captured by the imaging apparatus 90,the second images 81 to 84 that can be captured by the imaging apparatus90 can be captured by the imaging apparatus 90 by projecting the secondimages 81 to 84, and the projection of the projection apparatus 10 canbe adjusted based on the capturing results. Thus, the projection of theprojection apparatus 10 with respect to a wide projection range 11 canbe easily adjusted.

In addition, in repeating the control of projecting the second imagefrom the projection apparatus 10, the projection apparatus 10 mayperform a control of changing the second image projected from theprojection apparatus 10. For example, the projection apparatus 10 mayset the marker images included in the second image 81 as rectangles, setthe marker images included in the second image 82 as circles, set themarker images included in the second image 83 as triangles, and set themarker images included in the second image 84 as x marks. Accordingly,in receiving the captured images of the second images 81 to 84 from theimaging apparatus 90, the computer 50 can securely determine to which ofthe second images 81 to 84 the received captured images correspond bydetermining the shapes of the marker images included in the receivedcaptured images.

While processing of performing the imaging by sequentially projectingthe second images 81 to 84 is described, the present invention is notlimited thereto. For example, the computer 50 may project only thesecond image 81 among the second images 81 to 84 from the projectionapparatus 10. In this case, the computer 50 estimates the distortion ofthe projection image within the entire projection range 11 based on thedistortion in shape or arrangement of the marker images included in thesecond image 81 and performs the distortion correction of the projectionimage based on an estimation result.

In addition, while a case where the second images 81 to 84 are obtainedby extracting a part of the first image 71 is described, the secondimages 81 to 84 are not limited thereto. For example, in the examplesillustrated in FIG. 7 to FIG. 11 , the second images 81 to 84 may beimages in which 28 circular or triangular marker images are arranged ina 4×7 matrix. In addition, the second images 81 to 84 may be imagesincluding marker images having different shapes or colors.

Processing by Computer 50

FIG. 12 is a flowchart illustrating an example of processing by thecomputer 50. For example, the computer 50 executes the processingillustrated in FIG. 12 .

First, the computer 50 projects the first image 71 from the projectionapparatus 10 by transmitting a control signal to the projectionapparatus 10 (step S1201). For example, the computer 50 projects thefirst image 71 illustrated in FIG. 7 from the projection apparatus 10.

Next, the computer 50 performs the control of prompting the user of theimaging apparatus 90 to capture the first image 71 projected in stepS1201 by the imaging apparatus 90 (step S1202). For example, thiscontrol is performed by controlling the projection apparatus 10 toproject the message to the projection range 11.

Next, the computer 50 receives the captured image of the first image 71obtained by the imaging prompted in step S1202 from the imagingapparatus 90 (step S1203). The transmission of the captured image by theimaging apparatus 90 may be automatically performed by the imagingapparatus 90 with the imaging of the imaging apparatus 90 as a trigger,or may be performed by a user operation after the imaging of the imagingapparatus 90.

Next, the computer 50 determines whether or not all marker images of thefirst image 71 are included in the captured image received in step S1203(step S1204). In step S1204, in a case where all marker images areincluded (step S1204: Yes), the computer 50 performs the control ofadjusting the projection of the projection apparatus 10 based on thecaptured image of the first image 71 received in step S1203 (step S1205)and finishes the series of processing.

In step S1204, in a case where at least any of the marker images is notincluded (step S1204: No), the computer 50 generates a plurality ofsecond images (step S1206). For example, the computer 50 calculates theimageable range 72 based on the captured image of the first image 71received in step S1203 and generates second images (for example, thesecond images 81 to 84) that can cover the projection range 11 based onthe calculated imageable range 72.

Next, the computer 50 projects a non-projected second image from theprojection apparatus 10 among the second images generated in step S1206(step S1207). Next, the computer 50 performs the control of promptingthe user of the imaging apparatus 90 to capture the second imageprojected in step S1207 by the imaging apparatus 90 (step S1208).

Next, the computer 50 receives the captured image of the second imageobtained by the imaging prompted in step S1208 from the imagingapparatus 90 (step S1209). Next, the computer 50 determines whether ornot all second images generated in step S1206 are completely projectedin step S1207 (step S1210).

In step S1210, in a case where at least any of the second images is notprojected (step S1210: No), the computer 50 returns to step S1207. In acase where all second images are projected (step S1210: Yes), thecomputer 50 performs the control of adjusting the projection of theprojection apparatus 10 based on the captured images of the plurality ofsecond images received in step S1209 (step S1211) and finishes theseries of processing.

Another Example of First Image 71 Projected by Projection Apparatus 10

FIG. 13 is a diagram illustrating another example of the first image 71projected by the projection apparatus 10. As illustrated in FIG. 13 ,the first image 71 may be an image in which different numbers or thelike for each position are arranged as the marker images.

In addition, the computer 50 stores a correspondence table in which eachnumber included in the first image 71 as the marker image is associatedwith a position at which the number is arranged in the projection range11. That is, the marker images of the first image 71 illustrated in FIG.13 are associated with positions of the marker images within theprojection range 11.

Accordingly, even in a case where only a part of the marker images ofthe first image 71 is included in the captured image received from theimaging apparatus 90, the computer 50 can determine at which positionthe marker images of the part are present within the projection range11.

For example, in a case where marker images of a predetermined ratio ormore of the marker images of the first image 71 are included in thecaptured image of the first image 71 received from the imaging apparatus90, the computer 50 determines the position of each marker imageincluded in the captured image of the first image 71 within theprojection range 11 based on the correspondence table, detects thedistortion of the projection image within the entire projection range 11based on a determination result, and performs the distortion correctionof the projection image based on the detection result. In this case, thecomputer 50 may not perform the control of projecting the second image(for example, the second images 81 to 84) from the projection apparatus10.

Another Example of Second Image 81 Projected by Projection Apparatus 10

FIG. 14 is a diagram illustrating another example of the second image 81projected by the projection apparatus 10. As illustrated in FIG. 14 ,the second image 81 may be an image in which different numbers or thelike for each position are arranged as the marker images. In the examplein FIG. 14 , the second image 81 is obtained by extracting an upper leftpart of 4×7=28 marker images from the first image 71 illustrated in FIG.13 .

In addition, the computer 50 stores a correspondence table in which eachnumber included in the second image 81 as the marker image is associatedwith a position at which the number is arranged in the projection range11. This correspondence table may be the same as the correspondencetable described using FIG. 13 . That is, the marker images of the secondimage 81 illustrated in FIG. 14 are associated with positions of themarker images within the projection range 11.

Accordingly, even in a case where only a part of the second image 81 isincluded in the captured image received from the imaging apparatus 90,the computer 50 can determine at which position the marker images of thepart are present within the projection range 11.

While the second image 81 is described, the second images 82 to 84 mayalso be images in which different numbers or the like for each positionare arranged as the marker images.

For example, in a case where only a part of the marker images of thesecond image 81 is included in the captured image of the second image 81received from the imaging apparatus 90, the computer 50 determines theposition of each marker image included in the captured image of thesecond image 81 within the projection range 11 based on thecorrespondence table and combines the captured image of the second image81 with the captured images of the second images 82 to 84 based on thedetermination result. Accordingly, while a part of a region of thesecond image 81 is missing, an image equivalent to the captured image ofthe entire first image 71 can be generated. The computer 50 detects thedistortion of the projection image within the projection range 11 basedon the distortion in shape or arrangement of the marker images includedin the generated image and performs the distortion correction of theprojection image based on the detection result.

In the examples in FIG. 13 and FIG. 14 , while different numbers foreach position are illustratively described as the marker imagesassociated with the positions within the projection range 11, the markerimages associated with the positions within the projection range 11 arenot limited thereto and may be, for example, images of differentalphabets or symbols for each position. The marker images associatedwith the positions within the projection range 11 may be images ofdifferent colors or the like for each position. In addition, while anexample in which all marker images within the image are different isdescribed, only a part of the marker images in the image may bedifferent.

In addition, the marker images associated with the positions within theprojection range 11 may be different Quick Response (QR) codes(registered trademark), ArUco markers, or the like for each position. Inthis case, information indicating the positions of the marker imageswithin the projection range 11 can be included in the marker images.Accordingly, even in a case where the correspondence table is notstored, the computer 50 can determine the positions of the marker imageswithin the projection range 11 by reading the information of the QRcodes or ArUco markers included in the captured image as the markerimages.

Example of State Before Adjustment of Stack Projection by Plurality ofProjection Apparatuses

FIG. 15 is a diagram illustrating an example of a state beforeadjustment of stack projection by a plurality of projection apparatuses.In FIG. 15 , the same parts as the parts illustrated in FIG. 1 will bedesignated by the same reference numerals and will not be described. Asillustrated in FIG. 15 , the projection system 100 may further include aprojection apparatus 10A.

The projection apparatus 10A has the same configuration as theprojection apparatus 10 and performs projection to the projection targetobject 6 together with the projection apparatus 10. The computer 50 cancommunicate with the projection apparatus 10A. In the exampleillustrated in FIG. 15 , the computer 50 is connected to the projectionapparatus 10A through a communication cable 8A and can communicate withthe projection apparatus 10A.

A projection range 11A illustrated by a double dot dashed line is aregion that is irradiated with projection light by the projectionapparatus 10A in the projection target object 6. In the exampleillustrated in FIG. 15 , the projection range 11A is rectangular. Theprojection range 11A is a part or the entirety of a projectable rangewithin which the projection can be performed by the projection apparatus10A.

In this example, a case of performing the stack projection for improvinga dynamic range or gradation representation by making the entireprojection range 11 of the projection apparatus 10 overlap with theentire projection range 11A of the projection apparatus 10A andprojecting the same image from the projection apparatuses 10 and 10Awill be described.

Processing Based on Imaging Condition by Computer 50

FIG. 16 is a flowchart illustrating an example of processing by thecomputer 50 based on an imaging condition. In the example illustrated inFIG. 15 , for example, the computer 50 may execute the processingillustrated in FIG. 16 .

First, the computer 50 acquires an imaging condition of the imagingapparatus 90 (step S161). This imaging condition of the imagingapparatus 90 includes whether the imaging by the imaging apparatus 90 isfixed imaging or handheld imaging. The fixed imaging is imaging in astate where the imaging apparatus 90 is fixed to an object such as atripod or a seat that does not shake. The handheld imaging is imaging ina state where the imaging apparatus 90 is held in hands by the user, anda shake occurs.

For example, the computer 50 acquires the imaging condition of whetherthe imaging is the fixed imaging or the handheld imaging by receiving auser operation with respect to the computer 50. Alternatively, thecomputer 50 may acquire the imaging condition of whether the imaging isthe fixed imaging or the handheld imaging based on a state of the shakeof the imaging apparatus 90 depending on a gyro sensor or the like inthe imaging apparatus 90, by communicating with the imaging apparatus90. For example, in a case where the shake of the imaging apparatus 90detected by the gyro sensor or the like is less than a threshold value,the computer 50 determines that the imaging is the fixed imaging. In acase where the shake of the imaging apparatus 90 detected by the gyrosensor or the like is greater than or equal to the threshold value, thecomputer 50 determines that the imaging is the handheld imaging.

Alternatively, the computer 50 may acquire the imaging condition ofwhether the imaging is the fixed imaging or the handheld imaging basedon an ON/OFF state of a camera shake correction function in the imagingapparatus 90, by communicating with the imaging apparatus 90. Forexample, in a case where the camera shake correction function in theimaging apparatus 90 is in the OFF state, the computer 50 determinesthat the imaging is the fixed imaging. In a case where the camera shakecorrection function in the imaging apparatus 90 is in the ON state, thecomputer 50 determines that the imaging is the handheld imaging.

Next, the computer 50 determines whether or not the imaging of theimaging apparatus 90 is the handheld imaging based on the imagingcondition acquired in step S161 (step S162). In a case where the imagingis not the handheld imaging (step S162: No), the computer 50individually performs projection adjustment controls of the projectionapparatuses 10 and 10A (step S163) and finishes the series ofprocessing. The individual projection adjustment controls in step S163will be described using FIG. 17 and FIG. 18 .

In step S162, in a case where the imaging is the handheld imaging (stepS162: Yes), the computer 50 performs the projection adjustment controlsof the projection apparatuses 10 and 10A at the same time (step S164)and finishes the series of processing. The projection adjustmentcontrols in step S164 will be described using FIG. 19 .

Individual Projection of Projection Apparatuses 10 and 10A

FIG. 17 and FIG. 18 are diagrams illustrating an example of individualprojection of the projection apparatuses 10 and 10A. In a case where theimaging of the imaging apparatus 90 is the fixed imaging, for example,the computer 50 first adjusts the projection of the projection apparatus10 by executing the processing illustrated in FIG. 12 on the projectionapparatus 10 in step S163 in FIG. 16 . Next, the computer 50 adjusts theprojection of the projection apparatus 10A by executing the processingillustrated in FIG. 12 on the projection apparatus 10A.

FIG. 17 illustrates a state where the first image 71 is projected by theprojection apparatus 10 in step S1201 of the processing in FIG. 12executed on the projection apparatus 10. While illustration is notprovided, in a case where step S1207 is executed in the processing inFIG. 12 executed on the projection apparatus 10, the second images 81 to84 are projected from the projection apparatus 10 as in FIG. 8 to FIG.11 .

FIG. 18 illustrates a state where a first image 71A is projected by theprojection apparatus 10A in step S1201 of the processing in FIG. 12executed on the projection apparatus 10A. The first image 71Aillustrated in FIG. 18 is the same image as the first image 71illustrated in FIG. 17 . However, marker images included in the firstimage 71A are marker images that can be distinguished from the markerimages included in the first image 71 by the image recognition. Forexample, the marker images that can be distinguished by the imagerecognition are marker images having different shapes, different colors,different line types (a solid line, a dotted line, and the like), ordifferent spatial frequencies (arrangement intervals).

In the examples in FIG. 17 and FIG. 18 , while the marker imagesincluded in the first image 71 are rectangular white outlined images,the marker images included in the first image 71A are rectangular blacksolid images. While illustration is not provided, in a case where stepS1207 is executed in the processing in FIG. 12 executed on theprojection apparatus 10A, the second images are projected from theprojection apparatus 10A as in FIG. 8 to FIG. 11 . While these secondimages are the same as the second images 81 to 84, the rectangular blacksolid marker images are included instead of the rectangular whiteoutlined marker images.

As illustrated in FIG. 17 and FIG. 18 , in a case where the imaging ofthe imaging apparatus 90 is the fixed imaging, the computer 50individually performs the projection adjustment controls of theprojection apparatuses 10 and 10A. In this case, the computer 50projects the first images 71 and 71A from the projection apparatuses 10and 10A at different timings. However, since the imaging of the imagingapparatus 90 is the fixed imaging, a position and a direction of theimaging apparatus 90 are fixed even in a case where the first images 71and 71A are projected and captured at different timings. Thus, thecomputer 50 can determine a relative positional relationship between theprojection range 11 and the projection range 11A based on capturedimages of the first images 71 and 71A.

In addition, since the first images 71 and 71A are projected andcaptured at different timings, the computer 50 can receive each of thecaptured image of the first image 71 and the captured image of the firstimage 71A from the imaging apparatus 90. Thus, since the computer 50 maynot perform processing of separately extracting each of the first images71 and 71A from one captured image, the relative positional relationshipbetween the projection ranges 11 and 11A can be securely determined bysimple processing.

Projection of Projection Apparatuses 10 and 10A at Same Time

FIG. 19 is a diagram illustrating an example of projection of theprojection apparatuses 10 and 10A at the same time. In a case where theimaging of the imaging apparatus 90 is the handheld imaging, forexample, the computer 50 adjusts the projection of the projectionapparatuses 10 and 10A at the same time by collectively executing theprocessing illustrated in FIG. 12 on the projection apparatuses 10 and10A in step S164 in FIG. 16 .

FIG. 19 illustrates a state where the first images 71 and 71A areprojected at the same time by the projection apparatuses 10 and 10A instep S1201 of the processing in FIG. 12 executed on the projectionapparatuses 10 and 10A. In a case where step S1205 is executed in theprocessing in FIG. 12 executed on the projection apparatuses 10 and 10A,the computer 50 determines the relative positional relationship betweenthe projection range 11 and the projection range 11A by separatelyextracting each of the first image 71 from the projection apparatus 10and the first image 71A from the projection apparatus 10A from thecaptured image obtained in step S1203.

While illustration is not provided, in a case where step S1207 isexecuted in the processing in FIG. 12 executed on the projectionapparatuses 10 and 10A, first, the second image 81 from the projectionapparatus 10 illustrated in FIG. 8 and a second image (for example,obtained by extracting an upper left part of 4×7=28 marker images fromthe first image 71A) from the projection apparatus 10A are projected atthe same time.

Next, the second image 82 from the projection apparatus 10 illustratedin FIG. 9 and a second image (for example, obtained by extracting anupper right part of 4×7=28 marker images from the first image 71A) fromthe projection apparatus 10A are projected at the same time. Next, thesecond image 83 from the projection apparatus 10 illustrated in FIG. 10and a second image (for example, obtained by extracting a lower leftpart of 4×7=28 marker images from the first image 71A) from theprojection apparatus 10A are projected at the same time. Next, thesecond image 84 from the projection apparatus 10 illustrated in FIG. 11and a second image (for example, obtained by extracting a lower rightpart of 4×7=28 marker images from the first image 71A) from theprojection apparatus 10A are projected at the same time.

In step S1211 of the processing in FIG. 12 executed on the projectionapparatuses 10 and 10A, the computer 50 determines the relativepositional relationship between the projection range 11 and theprojection range 11A by separately extracting each of the second imagefrom the projection apparatus 10 and the second image from theprojection apparatus 10A from each captured image obtained in stepS1209.

As illustrated in FIG. 19 , in a case where the imaging of the imagingapparatus 90 is the handheld imaging, the computer 50 performs theprojection of the projection apparatuses 10 and 10A at the same time.Accordingly, even in a case where the position and the direction of theimaging apparatus 90 are changed by the handheld imaging, the relativepositional relationship between the projection range 11 and theprojection range 11A can be determined by separately extracting each ofthe image of the projection apparatus 10 and the image of the projectionapparatus 10A from the captured image including the image of theprojection apparatus 10 and the image of the projection apparatus 10A.In addition, since the first images 71 and 71A are projected andcaptured at the same time, the number of times the user of the imagingapparatus 90 performs the imaging can be reduced.

Stack Projection by Making Projection Ranges 11 and 11A Overlap

FIG. 20 is a diagram illustrating an example of the stack projection bymaking the projection ranges 11 and 11A overlap. As described using FIG.17 , FIG. 18 , or FIG. 19 , the computer 50 determines the relativepositional relationship between the projection ranges 11 and 11A.

The computer 50 adjusts relative projection positions between theprojection apparatus 10 and the projection apparatus 10A based on aresult of the determination such that the entire projection range 11overlaps with the entire projection range 11A as illustrated in FIG. 20. For example, this adjustment can be performed by controlling a shiftmechanism (the optical system shift mechanism or the electronic shiftmechanism) of at least any of the projection apparatus 10 or 10A.

For example, by controlling the shift mechanism of the projectionapparatus 10A to adjust the projection range 11A based on the projectionrange 11 of the projection apparatus 10, the computer 50 enables thestack projection by making the entire projection range 11 overlap withthe entire projection range 11A.

For example, in a case of using the electronic shift mechanism, thecomputer 50 calculates a conversion parameter for correcting theprojection range 11A such that the projection range 11A matches theprojection range 11. For example, the conversion parameter includes aprojective transformation (homography) matrix. The computer 50 can matchthe projection range 11A to the projection range 11 by correcting aninput image of the projection apparatus 10A using the calculatedconversion parameter and performing the projection from the projectionapparatus 10A.

In a case of performing the stack projection by making the projectionranges of the projection apparatuses 10 and 10A (the plurality ofprojection apparatuses) overlap, the computer 50 adjusts the overlappingbetween the projection range 11 and the projection range 11A byadjusting the projection range 11A of the projection apparatus 10A basedon the projection range 11 of the projection apparatus 10.

While a case of performing the projection adjustment controls for theprojection of the projection apparatuses 10 and 10A in the stackprojection of projecting the same image from the projection apparatuses10 and 10A by making the entire projection range 11 of the projectionapparatus 10 overlap with the entire projection range 11A of theprojection apparatus 10A is described, a form of performing theprojection adjustment controls for the projection of the projectionapparatuses 10 and 10A is not limited thereto.

For example, blending projection for obtaining a large screen ofprojection may be performed by making an end part of the projectionrange 11 of the projection apparatus 10 overlap with an end part of theprojection range 11A of the projection apparatus 10A and projecting eachof divided images obtained by dividing a large image from the projectionapparatuses 10 and 10A.

State Before Adjustment of Blending Projection by Plurality ofProjection Apparatuses

FIG. 21 is a diagram illustrating an example of a state beforeadjustment of the blending projection by the plurality of projectionapparatuses. In FIG. 21 , the same parts as the parts illustrated inFIG. 15 will be designated by the same reference numerals and will notbe described. In the example in FIG. 21 , in order to perform theblending projection, positions and directions of the projectionapparatuses 10 and 10A are adjusted such that only the end part of theprojection range 11 overlaps with the end part of the projection range11A.

Even in this case, as in the case of the stack projection, the computer50 can perform the projection adjustment controls in performingregistration or the distortion correction of the projection ranges 11and 11A. For example, in the state illustrated in FIG. 21 , the computer50 executes the processing illustrated in FIG. 16 . In this case, thefirst images and the second images projected from the projectionapparatuses 10 and 10A are the same as in the case of the stackprojection.

However, in this case, in adjusting the projection ranges 11 and 11A,the computer 50 adjusts a relative position between the projectionranges 11 and 11A such that a specific region (for example, a right endregion having a constant width) of the projection range 11 overlaps witha specific region (for example, a left end region having a constantwidth) of the projection range 11A. The specific region of theprojection range 11 and the specific region of the projection range 11Ahave the same size.

Furthermore, the computer 50 performs blending processing such asdividing brightness of each of the projection images from the projectionapparatuses 10 and 10A in half for an overlapping portion between theprojection ranges 11 and 11A. Accordingly, incongruity such as brightdisplay of only the overlapping portion between the projection ranges 11and 11A can be reduced.

Blending Projection by Making Projection Ranges 11 and 11A Overlap

FIG. 22 is a diagram illustrating an example of the blending projectionby making the projection ranges 11 and 11A overlap. In the processingillustrated in FIG. 16 , the computer 50 determines the relativepositional relationship between the projection ranges 11 and 11A basedon captured data from the imaging apparatus 90.

The computer 50 adjusts the relative projection positions between theprojection apparatus 10 and the projection apparatus 10A based on aresult of the determination such that the specific region of theprojection range 11 overlaps with the specific region of the projectionrange 11A as illustrated in FIG. 22 . For example, this adjustment canbe performed by controlling the shift mechanism (the optical systemshift mechanism or the electronic shift mechanism) of at least any ofthe projection apparatus 10 or 10A.

For example, by controlling the shift mechanisms of the projectionapparatuses 10 and 10A to adjust the projection ranges 11 and 11A, thecomputer 50 enables the blending projection by making the specificregion of the projection range 11 overlap with the specific region ofthe projection range 11A.

For example, in a case of using the electronic shift mechanisms, thecomputer 50 calculates a conversion parameter for correcting theprojection ranges 11 and 11A such that the specific region of theprojection range 11 matches the specific region of the projection range11A. For example, the conversion parameter includes a projectivetransformation (homography) matrix. The computer 50 can match thespecific region of the projection range 11 to the specific region of theprojection range 11A by correcting input images of the projectionapparatuses 10 and 10A using the calculated conversion parameter andperforming the projection from the projection apparatuses 10 and 10A.

In a case of performing the blending projection by making a part of theprojection ranges of the projection apparatuses 10 and 10A (theplurality of projection apparatuses) overlap, the computer 50 adjuststhe overlapping between the specific region of the projection range 11and the specific region of the projection range 11A by adjusting each ofthe projection ranges 11 and 11A. Accordingly, by performing theblending processing such as adjusting brightness of the specificregions, the blending processing is applied to only the overlappingportion between the projection range 11 and the projection range 11A,and incongruity of appearance can be reduced.

Another Example of Processing Based on Imaging Condition by Computer 50

FIG. 23 is a flowchart illustrating another example of the processing bythe computer 50 based on the imaging condition. The computer 50 mayexecute the processing illustrated in FIG. 23 . In this example, forexample, the projection apparatus 10 is the only projection apparatusthat performs the projection.

First, the computer 50 acquires the imaging condition of the imagingapparatus 90 (step S231). This imaging condition of the imagingapparatus 90 includes the resolution of the imaging of the imagingapparatus 90. The resolution is definition of the imaging and is decidedby the number of pixels of an imaging sensor of the imaging apparatus90, the angle of view of the imaging of the imaging apparatus 90, andthe like.

For example, the computer 50 acquires the resolution of the imaging byreceiving the resolution by the user operation with respect to thecomputer 50. Alternatively, the computer 50 may acquire the resolutionof the imaging by communicating with the imaging apparatus 90 to receiveinformation such as the number of pixels and the angle of view from theimaging apparatus 90.

Next, the computer 50 sets a size of the second image based on theresolution of the imaging included in the imaging condition acquired instep S231 (step S232). For example, in a case where the resolution ofthe imaging is greater than or equal to a threshold value, the computer50 sets the size of the second image to “normal”. In a case where theresolution of the imaging is less than the threshold value, the computer50 sets the size of the second image to “small”.

Next, the computer 50 performs the projection adjustment control of theprojection apparatus 10 (step S233) and finishes the series ofprocessing. In step S233, for example, the computer 50 adjusts theprojection of the projection apparatus 10 by executing the processingillustrated in FIG. 12 on the projection apparatus 10. In a case ofexecuting step S1206 in the processing illustrated in FIG. 12 , thecomputer 50 generates the second images based on the size of the secondimage set in step S232.

For example, in a case where the size of the second image is set to“normal” in step S232, the computer 50 generates the second images 81 to84 illustrated in FIG. 8 to FIG. 11 in step S1206. Meanwhile, in a casewhere the size of the second image is set to “small” in step S232, thecomputer 50 generates second images smaller than the second images 81 to84 illustrated in FIG. 8 to FIG. 11 in step S1206. Since these secondimages are smaller than the second images 81 to 84 illustrated in FIG. 8to FIG. 11 , the computer 50 generates more second images than thesecond images 81 to 84 illustrated in FIG. 8 to FIG. 11 to cover theprojection range 11.

Second Image in Case where Resolution of Imaging of Imaging Apparatus 90is Low

FIG. 24 is a diagram illustrating an example of the second image in acase where the resolution of the imaging of the imaging apparatus 90 islow. In a case where the size of the second image is set to “small” instep S232 in FIG. 23 , for example, the computer 50 projects the secondimage 81 illustrated in FIG. 24 from the projection apparatus 10 in stepS1206 of the first execution of the processing in FIG. 12 .

The second image 81 illustrated in FIG. 24 is a small image having asmall number of included marker images, compared to the second image 81illustrated in FIG. 8 . Accordingly, the user of the imaging apparatus90 can capture the second image 81 by approaching closer to theprojection target object 6. Thus, even in a case where the resolution ofthe imaging of the imaging apparatus 90 is low, a captured image fromwhich the marker images included in the second image 81 can be extractedwith high accuracy can be obtained.

While the second image 81 projected in step S1206 of the first executionof the processing in FIG. 12 is described in FIG. 24 , the computer 50performs a control of projecting the same second image as the secondimage 81 illustrated in FIG. 24 from the projection apparatus 10 whilechanging the projection position in step S1206 of the second and laterexecution of the processing in FIG. 12 .

As illustrated in FIG. 23 and FIG. 24 , the computer 50 may perform acontrol of projecting the second image having the size corresponding tothe resolution of the imaging of the imaging apparatus 90 from theprojection apparatus 10. Accordingly, even in a case where theresolution of the imaging of the imaging apparatus 90 is low, a capturedimage from which the marker images included in the second image can beextracted with high accuracy is obtained, and the projection of theprojection apparatus 10 can be accurately adjusted.

While the control of projecting the second image having the sizecorresponding to the resolution of the imaging from the projectionapparatus 10 in a case of performing the projection of only theprojection apparatus 10 is described, a control of projecting the secondimage having the size corresponding to the resolution of the imagingfrom the projection apparatuses 10 and 10A may also be performed in acase of performing the projection of the projection apparatuses 10 and10A.

Still Another Example of Processing Based on Imaging Condition byComputer 50

FIG. 25 is a flowchart illustrating still another example of theprocessing by the computer 50 based on the imaging condition. Thecomputer 50 may execute the processing illustrated in FIG. 25 . In thisexample, for example, the projection apparatus 10 is the only projectionapparatus that performs the projection.

First, the computer 50 acquires the imaging condition of the imagingapparatus 90 as in step S231 in FIG. 23 (step S251). Next, the computer50 sets sizes of the marker images included in the second image based onthe resolution of the imaging included in the imaging condition acquiredin step S251 (step S252). For example, in a case where the resolution ofthe imaging is greater than or equal to the threshold value, thecomputer 50 sets the sizes of the marker images included in the secondimage to “normal”. In a case where the resolution of the imaging is lessthan the threshold value, the computer 50 sets the sizes of the markerimages of the second image to “large”.

Next, the computer 50 performs the projection adjustment control of theprojection apparatus 10 (step S253) and finishes the series ofprocessing. In step S253, for example, the computer 50 adjusts theprojection of the projection apparatus 10 by executing the processingillustrated in FIG. 12 on the projection apparatus 10. In a case ofexecuting step S1206 in the processing illustrated in FIG. 12 , thecomputer 50 generates the second images based on the sizes of the markerimages set in step S252.

For example, in a case where the sizes of the marker images are set to“normal” in step S252, the computer 50 generates the second images 81 to84 illustrated in FIG. 8 to FIG. 11 in step S1206. Meanwhile, in a casewhere the sizes of the marker images are set to “large” in step S252,the computer 50 generates second images including larger marker imagesthan the second images 81 to 84 illustrated in FIG. 8 to FIG. 11 in stepS1206.

Marker Images of Second Image in Case where Resolution of Imaging ofImaging Apparatus 90 is Low

FIG. 26 is a diagram illustrating an example of the marker images of thesecond image in a case where the resolution of the imaging of theimaging apparatus 90 is low. In a case where the sizes of the markerimages are set to “large” in step S252 in FIG. 25 , for example, thecomputer 50 projects the second image 81 illustrated in FIG. 26 from theprojection apparatus 10 in step S1206 of the first execution of theprocessing in FIG. 12 .

The marker images of the second image 81 illustrated in FIG. 26 arelarge images, compared to the marker images of the second image 81illustrated in FIG. 8 . Accordingly, even in a case where the resolutionof the imaging of the imaging apparatus 90 is low, the captured image ofthe second image 81 including the marker images that can be extractedwith higher accuracy because the marker images are large can beobtained.

While the second image 81 projected in step S1206 of the first executionof the processing in FIG. 12 is described in FIG. 26 , the computer 50performs a control of projecting the same second image as the secondimage 81 illustrated in FIG. 26 from the projection apparatus 10 whilechanging the projection position in step S1206 of the second and laterexecution of the processing in FIG. 12 .

As illustrated in FIG. 25 and FIG. 26 , the computer 50 may perform acontrol of projecting the second image including the marker imageshaving the sizes corresponding to the resolution of the imaging of theimaging apparatus 90 from the projection apparatus 10. Accordingly, evenin a case where the resolution of the imaging of the imaging apparatus90 is low, a captured image including the marker images that can beextracted with higher accuracy is obtained, and the projection of theprojection apparatus 10 can be accurately adjusted.

While the control of projecting the second image including the markerimages having the sizes corresponding to the resolution of the imagingin a case of performing the projection of only the projection apparatus10 is described, a control of projecting the second image including themarker images having the sizes corresponding to the resolution of theimaging from the projection apparatuses 10 and 10A may also be performedin a case of performing the projection of the projection apparatuses 10and 10A.

Modification Example 1

While a configuration in which the optical axis K is not bent isdescribed as a configuration of the projection apparatus 10 in FIG. 4and FIG. 5 , the optical axis K may be configured to be bent once ormore by providing a reflective member in the optical unit 106.

FIG. 27 is a schematic diagram illustrating another exteriorconfiguration of the projection apparatus 10. FIG. 28 is a schematiccross-sectional view of the optical unit 106 of the projection apparatus10 illustrated in FIG. 27 . In FIG. 27 and FIG. 28 , the same parts asthe parts illustrated in FIG. 4 and FIG. 5 will be designated by thesame reference numerals and will not be described.

As illustrated in FIG. 27 , the optical unit 106 comprises a secondmember 103 supported by the first member 102 in addition to the firstmember 102 supported by the body part 101. The first member 102 and thesecond member 103 may be an integrated member.

As illustrated in FIG. 28 , the optical unit 106 comprises, in additionto the first member 102, the second member 103 including a hollowportion 3A connected to the hollow portion 2A of the first member 102,the first optical system 121 and a reflective member 122 arranged in thehollow portion 2A, a second optical system 31, a reflective member 32, athird optical system 33, and the lens 34 arranged in the hollow portion3A, the first shift mechanism 105, and a projection direction changingmechanism 104.

In the examples in FIG. 27 and FIG. 28 , the opening 2 a and the opening2 b of the first member 102 are formed in surfaces perpendicular to eachother. In addition, the projection optical system 23 illustrated in FIG.27 and FIG. 28 is composed of the reflective member 122, the secondoptical system 31, the reflective member 32, and the third opticalsystem 33 in addition to the first optical system 121 and the lens 34illustrated in FIG. 4 and FIG. 5 . This projection optical system 23forms the optical axis K to be folded by being bent twice as illustratedin FIG. 28 . The first optical system 121, the reflective member 122,the second optical system 31, the reflective member 32, the thirdoptical system 33, and the lens 34 are arranged in this order from thelight modulation portion 22 side along the optical axis K.

The first optical system 121 guides the light that is incident on thefirst member 102 from the body part 101 and travels in the direction X1,to the reflective member 122. The reflective member 122 reflects thelight incident from the first optical system 121 in the direction Y1.The reflective member 122 is configured with, for example, a mirror. Inthe first member 102, the opening 2 b is formed on the optical path ofthe light reflected by the reflective member 122, and the reflectedlight travels to the hollow portion 3A of the second member 103 bypassing through the opening 2 b.

The second member 103 is a member having an approximately L-shapedcross-sectional exterior, in which an opening 3 a is formed at aposition facing the opening 2 b of the first member 102. The light thathas passed through the opening 2 b of the first member 102 from the bodypart 101 is incident into the hollow portion 3A of the second member 103through the opening 3 a. The first member 102 and the second member 103may have any cross-sectional exterior and are not limited to the above.

The second optical system 31 includes at least one lens and guides thelight incident from the first member 102 to the reflective member 32.The reflective member 32 guides the light incident from the secondoptical system 31 to the third optical system 33 by reflecting the lightin the direction X2. The reflective member 32 is configured with, forexample, a mirror. The third optical system 33 includes at least onelens and guides the light reflected by the reflective member 32 to thelens 34.

The lens 34 is arranged in an end part of the second member 103 on thedirection X2 side in the form of closing the opening 3 c formed in thisend part. The lens 34 projects the light incident from the third opticalsystem 33 to the projection target object 6.

FIG. 28 illustrates a state where the first member 102 is moved as faras possible to the direction Y1 side by the first shift mechanism 105.By moving the first member 102 in the direction Y2 by the first shiftmechanism 105 from the state illustrated in FIG. 28 , a relativeposition between a center of the image formed by the light modulationportion 22 and the optical axis K changes, and the image G1 projected tothe projection target object 6 can be shifted in the direction Y1.

The projection direction changing mechanism 104 is a rotation mechanismthat rotatably connects the second member 103 to the first member 102.By the projection direction changing mechanism 104, the second member103 is configured to be rotatable about a rotation axis (specifically,the optical axis K) that extends in the direction Y. The projectiondirection changing mechanism 104 is not limited to an arrangementposition illustrated in FIG. 28 as long as the projection directionchanging mechanism 104 can rotate the optical system. In addition, thenumber of rotation mechanisms is not limited to one, and a plurality ofrotation mechanisms may be provided.

Modification Example 2

While the computer 50 is illustratively described as an example of thecontrol device according to the embodiment of the present invention, thecontrol device according to the embodiment of the present invention isnot limited thereto. For example, the control device according to theembodiment of the present invention may be the projection apparatus 10(or the projection apparatus 10A). In this case, each control of thecomputer 50 is performed by the projection apparatus 10. The projectionapparatus 10 may communicate with the imaging apparatus 90 through thecomputer 50 or may communicate with the imaging apparatus 90 withoutpassing through the computer 50. In a case where the projectionapparatus 10 communicates with the imaging apparatus 90 without passingthrough the computer 50, the computer 50 may be configured to be omittedfrom the projection system 100.

Alternatively, the control device according to the embodiment of thepresent invention may be the imaging apparatus 90. In this case, eachcontrol of the computer 50 is performed by the imaging apparatus 90. Theimaging apparatus 90 may communicate with the projection apparatuses 10and 10A through the computer 50 or may communicate with the projectionapparatuses 10 and 10A without passing through the computer 50. In acase where the imaging apparatus 90 communicates with the projectionapparatuses 10 and 10A without passing through the computer 50, thecomputer 50 may be configured to be omitted from the projection system100.

Modification Example 3

While the projection apparatuses 10 and 10A are illustratively describedas an example of the plurality of projection apparatuses, the pluralityof projection apparatuses may be three or more projection apparatuses.

At least the following matters are disclosed in the presentspecification.

(1) A control device of a projection system including one or moreprojection apparatuses that project a first image including a pluralityof marker images, and an imaging apparatus that captures at least a partof the first image, the control device comprising a processor, in whichthe processor is configured to perform a control of projecting a secondimage including a plurality of marker images from the projectionapparatus based on a capturing result of at least the part of the firstimage by the imaging apparatus.

(2) The control device according to (1), in which the processor isconfigured to perform a control of adjusting the projection of theprojection apparatus based on a capturing result of the second image bythe imaging apparatus.

(3) The control device according to (1) or (2), in which one or moremarker images among the marker images are images associated withpositions of the marker images within a projection range of theprojection apparatus.

(4) The control device according to any one of (1) to (3), furthercomprising a correspondence table in which the marker images areassociated with positions of the marker images within a projection rangeof the projection apparatus, in which the processor is configured todetermine the positions of the marker images within the projection rangeof the projection apparatus based on the correspondence table.

(5) The control device according to any one of (1) to (4), in which theprocessor is configured to calculate an imageable range of the imagingapparatus based on the capturing result of at least the part of thefirst image and perform the control of projecting the second image basedon the imageable range from the projection apparatus.

(6) The control device according to (5), in which the processor isconfigured to repeat the control of projecting the second image from theprojection apparatus by generating the second image different from thefirst image based on the imageable range.

(7) The control device according to (6), in which the processor isconfigured to, in a case of repeating the control of projecting thesecond image from the projection apparatus, perform a control ofchanging a projection position of the second image.

(8) The control device according to (6) or (7), in which the processoris configured to, in a case of repeating the control of projecting thesecond image from the projection apparatus, perform a control ofchanging the second image projected from the projection apparatus.

(9) The control device according to any one of (1) to (8), in which theprocessor is configured to acquire an imaging condition of the imagingapparatus and perform the control of projecting the second image fromthe projection apparatus based on the acquired imaging condition.

(10) The control device according to (9), in which the projectionapparatus includes a plurality of projection apparatuses, the imagingcondition includes whether the imaging of the imaging apparatus is fixedimaging or handheld imaging, and the processor is configured to, in acase where the imaging is the fixed imaging, perform the control ofprojecting at least any of the first image or the second image from theplurality of projection apparatuses at different timings.

(11) The control device according to (10), in which the processor isconfigured to, in a case where the imaging is the handheld imaging,perform the control of projecting distinguishable images of at least anyof the first image or the second image from the plurality of projectionapparatuses at the same time.

(12) The control device according to any one of (9) to (11), in whichthe imaging condition includes a resolution of the imaging, and theprocessor is configured to perform the control of projecting the secondimage having a size corresponding to the resolution from the projectionapparatus.

(13) The control device according to any one of (9) to (11), in whichthe imaging condition includes a resolution of the imaging, and theprocessor is configured to perform the control of projecting the secondimage including the marker images having sizes corresponding to theresolution from the projection apparatus.

(14) The control device according to any one of (1) to (13), in whichthe projection apparatus includes a plurality of projection apparatusesof which at least parts of projection ranges overlap, and the processoris configured to perform a control of adjusting the overlapping.

(15) The control device according to (14), in which the processor isconfigured to, based on a projection range of a first projectionapparatus included in the plurality of projection apparatuses, performthe control of adjusting the overlapping by adjusting a projection rangeof a second projection apparatus that is included in the plurality ofprojection apparatuses and is different from the first projectionapparatus.

(16) The control device according to (15), in which the processor isconfigured to, in a case of making the projection ranges of theplurality of projection apparatuses overlap, perform the control ofadjusting the overlapping by adjusting the projection range of thesecond projection apparatus based on the projection range of the firstprojection apparatus.

(17) The control device according to (14), in which the processor isconfigured to perform the control of adjusting the overlapping byadjusting projection images of the plurality of projection apparatuses.

(18) The control device according to (17), in which the processor isconfigured to, in a case of making parts of the projection ranges of theplurality of projection apparatuses overlap, perform the control ofadjusting the overlapping by adjusting the projection ranges of theplurality of projection apparatuses.

(19) A control method by a control device of a projection systemincluding one or more projection apparatuses that project a first imageincluding a plurality of marker images, and an imaging apparatus thatcaptures at least a part of the first image, the control deviceincluding a processor, the control method comprising performing, by theprocessor, a control of projecting a second image including a pluralityof marker images from the projection apparatus based on a capturingresult of at least the part of the first image by the imaging apparatus.

(20) The control method according to (19), in which the processorperforms a control of adjusting the projection of the projectionapparatus based on a capturing result of the second image by the imagingapparatus.

(21) The control method according to (19) or (20), in which one or moremarker images among the marker images are images associated withpositions of the marker images within a projection range of theprojection apparatus.

(22) The control method according to any one of (19) to (21), in which acorrespondence table in which the marker images are associated withpositions of the marker images within a projection range of theprojection apparatus is further provided, and the processor determinesthe positions of the marker images within the projection range of theprojection apparatus based on the correspondence table.

(23) The control method according to any one of (19) to (22), in whichthe processor calculates an imageable range of the imaging apparatusbased on the capturing result of at least the part of the first imageand performs the control of projecting the second image based on theimageable range from the projection apparatus.

(24) The control method according to (23), in which the processorrepeats the control of projecting the second image from the projectionapparatus by generating the second image different from the first imagebased on the imageable range.

(25) The control method according to (24), in which the processorperforms, in a case of repeating the control of projecting the secondimage from the projection apparatus, control of changing a projectionposition of the second image.

(26) The control method according to (24) or (25), in which theprocessor performs, in a case of repeating the control of projecting thesecond image from the projection apparatus, a control of changing thesecond image projected from the projection apparatus.

(27) The control method according to any one of (19) to (26), in whichthe processor acquires an imaging condition of the imaging apparatus andperforms the control of projecting the second image from the projectionapparatus based on the acquired imaging condition.

(28) The control method according to (27), in which the projectionapparatus includes a plurality of projection apparatuses, the imagingcondition includes whether the imaging of the imaging apparatus is fixedimaging or handheld imaging, and the processor performs, in a case wherethe imaging is the fixed imaging, the control of projecting at least anyof the first image or the second image from the plurality of projectionapparatuses at different timings.

(29) The control method according to (28), in which the processorperforms, in a case where the imaging is the handheld imaging, thecontrol of projecting distinguishable images of at least any of thefirst image or the second image from the plurality of projectionapparatuses at the same time.

(30) The control method according to any one of (27) to (29), in whichthe imaging condition includes a resolution of the imaging, and theprocessor performs the control of projecting the second image having asize corresponding to the resolution from the projection apparatus.

(31) The control method according to any one of (27) to (29), in whichthe imaging condition includes a resolution of the imaging, and theprocessor performs the control of projecting the second image includingthe marker images having sizes corresponding to the resolution from theprojection apparatus.

(32) The control method according to any one of (19) to (31), in whichthe projection apparatus includes a plurality of projection apparatusesof which at least parts of projection ranges overlap, and the processorperforms a control of adjusting the overlapping.

(33) The control method according to (32), in which the processorperforms, based on a projection range of a first projection apparatusincluded in the plurality of projection apparatuses, the control ofadjusting the overlapping by adjusting a projection range of a secondprojection apparatus that is included in the plurality of projectionapparatuses and is different from the first projection apparatus.

(34) The control method according to (33), in which the processorperforms, in a case of making the projection ranges of the plurality ofprojection apparatuses overlap, the control of adjusting the overlappingby adjusting the projection range of the second projection apparatusbased on the projection range of the first projection apparatus.

(35) The control method according to (32), in which the processorperforms the control of adjusting the overlapping by adjustingprojection images of the plurality of projection apparatuses.

(36) The control method according to (35), in which the processorperforms, in a case of making parts of the projection ranges of theplurality of projection apparatuses overlap, the control of adjustingthe overlapping by adjusting the projection ranges of the plurality ofprojection apparatuses.

(37) A projection system comprising one or more projection apparatusesthat project a first image including a plurality of marker images, animaging apparatus that captures at least a part of the first image, anda control device, in which the control device includes a processor, andthe processor is configured to perform a control of projecting a secondimage including a plurality of marker images from the projectionapparatus based on a capturing result of at least the part of the firstimage by the imaging apparatus.

(38) A control program causing a processor of a control device of aprojection system to execute a process, the projection system includingone or more projection apparatuses that project a first image includinga plurality of marker images, and an imaging apparatus that captures atleast a part of the first image, the process comprising performing acontrol of projecting a second image including a plurality of markerimages from the projection apparatus based on a capturing result of atleast the part of the first image by the imaging apparatus.

EXPLANATION OF REFERENCES

-   -   1: projection portion    -   2: operation reception portion    -   2A, 3A: hollow portion    -   2 a, 2 b, 3 a, 3 c, 15 a: opening    -   4: control portion    -   4 a: storage medium    -   5: communication portion    -   6: projection target object    -   8, 8A, 9: communication cable    -   10, 10A: projection apparatus    -   11, 11A: projection range    -   12: light modulation unit    -   15: housing    -   21: light source    -   22: light modulation portion    -   23: projection optical system    -   24: control circuit    -   31: second optical system    -   32, 122: reflective member    -   33: third optical system    -   34: lens    -   50: computer    -   51: processor    -   52: memory    -   53: communication interface    -   54: user interface    -   59: bus    -   71, 71A: first image    -   72: imageable range    -   81 to 84: second image    -   90: imaging apparatus    -   100: projection system    -   101: body part    -   102: first member    -   103: second member    -   104: projection direction changing mechanism    -   105: first shift mechanism    -   106: optical unit    -   121: first optical system    -   G1: image

What is claimed is:
 1. A control device of a projection system includingone or more projection apparatuses that project a first image includinga plurality of marker images, and an imaging apparatus that captures atleast a part of the first image, the control device comprising: aprocessor, wherein the processor is configured to perform a control ofprojecting a second image including a plurality of marker images fromthe projection apparatus based on a capturing result of at least thepart of the first image by the imaging apparatus.
 2. The control deviceaccording to claim 1, wherein the processor is configured to perform acontrol of adjusting the projection of the projection apparatus based ona capturing result of the second image by the imaging apparatus.
 3. Thecontrol device according to claim 1, wherein one or more marker imagesamong the marker images are images associated with positions of themarker images within a projection range of the projection apparatus. 4.The control device according to claim 1, further comprising: acorrespondence table in which the marker images are associated withpositions of the marker images within a projection range of theprojection apparatus, wherein the processor is configured to determinethe positions of the marker images within the projection range of theprojection apparatus based on the correspondence table.
 5. The controldevice according to claim 1, wherein the processor is configured tocalculate an imageable range of the imaging apparatus based on thecapturing result of at least the part of the first image and perform thecontrol of projecting the second image based on the imageable range fromthe projection apparatus.
 6. The control device according to claim 5,wherein the processor is configured to repeat the control of projectingthe second image from the projection apparatus by generating the secondimage different from the first image based on the imageable range. 7.The control device according to claim 6, wherein the processor isconfigured to, in a case of repeating the control of projecting thesecond image from the projection apparatus, perform a control ofchanging a projection position of the second image.
 8. The controldevice according to claim 6, wherein the processor is configured to, ina case of repeating the control of projecting the second image from theprojection apparatus, perform a control of changing the second imageprojected from the projection apparatus.
 9. The control device accordingto claim 1, wherein the processor is configured to acquire an imagingcondition of the imaging apparatus and perform the control of projectingthe second image from the projection apparatus based on the acquiredimaging condition.
 10. The control device according to claim 9, whereinthe projection apparatus includes a plurality of projection apparatuses,the imaging condition includes whether the imaging of the imagingapparatus is fixed imaging or handheld imaging, and the processor isconfigured to, in a case where the imaging is the fixed imaging, performthe control of projecting at least any of the first image or the secondimage from the plurality of projection apparatuses at different timings.11. The control device according to claim 10, wherein the processor isconfigured to, in a case where the imaging is the handheld imaging,perform the control of projecting distinguishable images of at least anyof the first image or the second image from the plurality of projectionapparatuses at a same time.
 12. The control device according to claim 9,wherein the imaging condition includes a resolution of the imaging, andthe processor is configured to perform the control of projecting thesecond image having a size corresponding to the resolution from theprojection apparatus.
 13. The control device according to claim 9,wherein the imaging condition includes a resolution of the imaging, andthe processor is configured to perform the control of projecting thesecond image including the marker images having sizes corresponding tothe resolution from the projection apparatus.
 14. The control deviceaccording to claim 1 wherein the projection apparatus includes aplurality of projection apparatuses of which at least parts ofprojection ranges overlap, and the processor is configured to perform acontrol of adjusting the overlapping.
 15. The control device accordingto claim 14, wherein the processor is configured to, based on aprojection range of a first projection apparatus included in theplurality of projection apparatuses, perform the control of adjustingthe overlapping by adjusting a projection range of a second projectionapparatus that is included in the plurality of projection apparatusesand is different from the first projection apparatus.
 16. The controldevice according to claim 15, wherein the processor is configured to, ina case of making the projection ranges of the plurality of projectionapparatuses overlap, perform the control of adjusting the overlapping byadjusting the projection range of the second projection apparatus basedon the projection range of the first projection apparatus.
 17. Thecontrol device according to claim 14, wherein the processor isconfigured to perform the control of adjusting the overlapping byadjusting projection images of the plurality of projection apparatuses.18. The control device according to claim 17, wherein the processor isconfigured to, in a case of making parts of the projection ranges of theplurality of projection apparatuses overlap, perform the control ofadjusting the overlapping by adjusting the projection ranges of theplurality of projection apparatuses.
 19. A control method by a controldevice of a projection system including one or more projectionapparatuses that project a first image including a plurality of markerimages, and an imaging apparatus that captures at least a part of thefirst image, the control device including a processor, the controlmethod comprising: performing, by the processor, a control of projectinga second image including a plurality of marker images from theprojection apparatus based on a capturing result of at least the part ofthe first image by the imaging apparatus.
 20. A projection systemcomprising: one or more projection apparatuses that project a firstimage including a plurality of marker images; an imaging apparatus thatcaptures at least a part of the first image; and a control device,wherein the control device includes a processor, and the processor isconfigured to perform a control of projecting a second image including aplurality of marker images from the projection apparatus based on acapturing result of at least the part of the first image by the imagingapparatus.
 21. A non-transitory computer readable medium storing acontrol program causing a processor of a control device of a projectionsystem to execute a process, the projection system including one or moreprojection apparatuses that project a first image including a pluralityof marker images, and an imaging apparatus that captures at least a partof the first image, the process comprising: performing a control ofprojecting a second image including a plurality of marker images fromthe projection apparatus based on a capturing result of at least thepart of the first image by the imaging apparatus.